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Abstract 

A dimension extractor is an algorithm designed to increase the effective dimen- 
sion - i.e., the amount of computational randomness - of an infinite binary sequence, 
in order to turn a "partially random" sequence into a "more random" sequence. Ex- 
tractors are exhibited for various effective dimensions, including constructive, com- 
putable, space-bounded, time-bounded, and finite-state dimension. Using similar 
techniques, the Kucera-Gacs theorem is examined from the perspective of decompres- 
sion, by showing that every infinite sequence S is Turing reducible to a Martin-Lof 
random sequence R such that the asymptotic number of bits of R needed to compute 
n bits of S, divided by n, is precisely the constructive dimension of S, which is shown 
to be the optimal ratio of query bits to computed bits achievable with Turing reduc- 
tions. The extractors and decompressors that are developed lead directly to new 
characterizations of some effective dimensions in terms of optimal decompression by 
Turing reductions. 



1 Introduction 

Effective dimension [33,34] and strong dimension [2] are effectivizations of classical Haus- 
dorff [21] and packing [49, 50] dimension, which can each be characterized in terms of 
betting strategies called martingales. By placing resource bounds on the martingales, indi- 
vidual infinite binary sequences can be assigned a non-zero effective dimension, interpreted 
as the density of computational randomness of the sequence. A dimension extractor is an 
algorithm designed to increase the effective dimension of a sequence. Like their counter- 
parts in computational complexity theory [46] , dimension extractors transform a source of 
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weak randomness into a source of strong randomness, the difference being that the algo- 
rithmic randomness of a sequence is being extracted, rather than classical randomness of 

a probabilistic source. 

To attack (but not settle) a question raised by Reimann [41], Terwijn, Miller, and 
Nies [38] concerning the abihty of Turing reductions to increase the constructive dimen- 
sion [34] of a sequence, we exhibit a constructive dimension extractor by showing that every 
sequence of positive constructive dimension is Turing equivalent to a sequence of constiTic- 
tive strong dimension [2] arbitrarily close to 1. The reduction is uniform with respect to 
the input sequence: a single oracle Turing machine, taking as input a rational upper bound 
on the dimension of the input sequence, works for every input sequence of positive con- 
structive dimension. The construction of this extractor follows in a straightforward manner 
from an earlier result of Ryabko [42,43] on constructive dimension and compression, and 
from Mayordomo's [36] characterization of constructive dimension in terms of Kolmogorov 
complexity (cf. [30]). 

We then develop new techniques to extend and improve this result for effective di- 
mensions at lower levels of computability; in particular, our first main result shows that 
computable, pjspace, and pj dimension [33] can be extracted using truth-table reductions, 
PjSpace Turing reductions, and pj Turing reductions, respectively. The p^ hierarchy was 
defined by Lutz [31] as a hierarchy of classes of functions computable in super-polynomial, 
but sub-exponential, space or time. For instance, pispace is simply pspace, the class of 
functions computable in space n*^ for some constant k, and p2space is the class of func- 
tions computable in space n*-^°^'*^ . It is also shown that finite-state dimension [2,12] can 
be extracted with information lossless finite-state transducers [26]. Thus, with respect to 
constructive, computable, pjspace, pj, and finite-state information density, any sequence 
in which almost every prefix has information density bounded away from zero can be used 
to compute a sequence in which infinitely many prefixes have information density that is 
nearly maximal. Furthermore, in the case of all dimensions except constructive dimension, 
in addition to the strong dimension extraction, a lower bound of rf/D — e is derived on the 
dimension of the extracted sequence, where d and D are the dimension and strong dimen- 
sion, respectively, of the input sequence, and e is an arbitrarily small positive constant. 
It follows that for any regular sequence (a sequence in which the dimension and strong 
dimension agree), both strong dimension and dimension are nearly optimally extracted. 

For all dimensions, the intuition behind the proof is the same. The extractor acts 
as a compressor that compresses the input sequence close to its optimal compression ratio 
under the resource bound A, which is precisely the A-dimension of the sequence. It is well- 
known [30] that the shortest program to produce a finite string must itself be incompressible 
(i.e., have maximal Kolmogorov complexity). Mayordomo's Kolmogorov complexity char- 
acterization of constructive dimension [36], Hitchcock's A-bounded Kolmogorov complexity 
characterization of A-dimension for A = comp or pispace [22], Lopez- Valdes and Mayor- 
domo's Pj compressor/decompressor characterization of p, dimension [51], and Dai, Lath- 
rop, Lutz, and Mayordomo's compression characterization of finite-state dimension [12], 
are invoked to show that a compressed representation of a sequence must itself be more in- 
compressible than the sequence and thus have higher dimension. Of course, this technique 
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works for any dimension that has a characterization in terms of Kolmogorov complexity, 
such as, for instance, exponential time or exponential space bounded dimension. 

In each case, the extractor is no more powerful than the resource bound defining the 
dimension, which is necessary to make the results non-trivial. For instance, without access 
to any oracle sequence, but given exponential space, a program can diagonalize against 
all pspace-bounded martingales to compute a pspace-random sequence. An extractor is 
interesting only when it has no more computational power than the class of algorithms it 
is trying to fool: all the randomness present in the output sequence must originate from 
the input sequence, and the extractor merely acts as a filter that distills the randomness 
out from the redundancy. 

We then examine the Kucera-Gacs theorem from the perspective of decompression. 
Kucera [28, 29] and Gacs [18] independently showed that every infinite sequence is Turing 
reducible to a Martin-Lof random sequence. Our second main result extends this theorem 
by showing that every infinite sequence S is Turing reducible to a Martin-Lof random 
sequence R such that the asymptotic best-case and worst case number of bits of R needed to 
compute n bits of S, divided by n, are precisely the constructive dimension and constructive 
strong dimension, respectively, of 5*. We show that this is the optimal ratio of query bits 
to computed bits achievable with Turing reductions. 

As an application of these results and techniques, the resource-bounded extractors 
are used to characterize the computable dimension of individual sequences in terms of 
decompression via truth-table reductions and to characterize the p^space dimension of 
individual sequences in terms of decompression via p^space-bounded Turing reductions, 
and the optimal decompression result is used to characterize constructive dimension in 
terms of decompression via Turing reductions. 

The paper is organized as follows. Section [2] explains notation and background material. 
Section [3] introduces the concepts of compression and decompression via reductions, and 
also gives definitions and background for effective dimension. Section H] exhibits dimension 
extractors for various effective dimensions. Section [5l which appeared in preliminary form 
as [13], uses techniques developed in section H] to show that every sequence is optimally 
decompressible from a random one. Section [H] uses results of sections S] and O to prove new 
characterizations of constructive, computable, and space-bounded dimensions. 

2 Preliminaries 

We refer the reader to [30] for an introduction to Kolmogorov complexity and algorith- 
mic information theory, [48] for an introduction to computability theory, and [40] for an 
introduction to computational complexity theory. 

2.1 Notation 

All logarithms are base 2. We write M, Q, Q2, and N for the set of all reals, rationals, 
dyadic rationals, integers, and non-negative integers, respectively. For all ACM, A'^ 



3 



denotes An(0,oo). {0,1}* denotes the set of all finite, binary strings. For all x G {0, 1}*, 

denotes the length of x. A denotes the empty string. Let Sq, Si, • • • G {0, 1}* denote 
the standard enumeration of binary strings sq = A, Si = 0, S2 = 1, S3 = 00, .... For A; G N, 
{0, 1}^ denotes the set of all strings x G {0, 1}* such that \x\ = k. C = {0, 1}°° denotes the 
Cantor space, the set of all infinite, binary sequences. For x G {0, 1}* and y G {0,1}*UC, 
xy denotes the concatenation of x and y, x C. y denotes that x is a prefix of y; i.e., there 
exists u G {0, 1}* U C such that xu = y, and x \Z y denotes that x \Z y and x ^ y. For 
5 G {0, 1}* U C and i,j G N, S\i] denotes the i*^' bit of S, with S[0] being the leftmost bit, 
S[i . . j] denotes the substring consisting of the i^^ through j^^ bits of S (inclusive), with 
S[i . . j] = A if i > j, and S \ i denotes S[0 . .i — 1]. A language is a subset of {0, 1}*, and 
we identify a language L C {0, 1}* with its characteristic sequence xl £ C, where the n*^ 
bit of Xl is 1 if and only if s„ G L, writing L \ i to denote xl \ i- 

2.2 Kolmogorov Complexity and Coding 

Fix a universal self-delimiting Turing machine U. Let w G {0, 1}*. The Kolmogorov 
complexity of w is 

K(w)— min { IttI \ UM = w} . 

7re{0,l}* 

The quantity is called the Kolmogorov rate of w. Given a time- and space-constructible 
bound i : N — > N, the t -time-bounded Kolmogorov complexity of w is 

K*(w) = min { IttI | UIt:) = w in at most t(\w\) time} , 

7re{0,l}* 

and the t- space-hounded Kolmogorov complexity of w is 

KS*(t(;) — min { |7r| | U{'k) — w in at most t{\w\) space} . 

7rG{0,l}* 

Fact 2.1. For all w G {0, 1}* andt-.n^n, K{w) < KS*H < K*(w). 

For all g G Q, let K{q) = K(6g), K\q) = K*(fog) and KS*(g) = KS*(6g), where 
hq G {0, 1}* is some standard binary representation of the rational q with a numerator, 

denominator, and sign bit. 

Define the self- delimiting encoding function enc : (0, 1}* — > {0, 1}* for all w G (0, 1}* 

by 

enc(w) = 0*i"'ils|i„|W. 

For all n G N, let enc(n) = enc(s„). Strings encoded by enc and valid programs for U are 
self- delimiting. They can be prepended to arbitrary strings and uniquely decoded. 

Observation 2.2. For all w G {0, 1}*, |enc(w)| < |w| -|- 2 log |w| -|- 3, and for all n e N 
such that n >2, enc(n) < logn -|- 2 log log n -|- 3. 
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Our results, being asymptotic in nature, do not depend crucially on using the self- 
delimiting Kolmogorov complexity K; it is simply more convenient for encoding purposes. 
All results in this paper hold if we use the plain Kolmogorov complexity C : {0, 1}* ^ N 
(see [30]) instead. Whenever we would need to add a program to a string and retain the 
ability to uniquely decode it, we could simply encode the program using the function enc. 

2.3 Space/Time Bounds 

The following explanation of growth rates and function classes is taken nearly verbatim 
from [31]. 

For each i & N, define a class Gj of growth rates between linear and exponential as 
follows. 

Go = { i : N ^ N I (3A; e N)(V~n e N) t{n) < kn } 
Gi+i = 2^'(i°sn) = I t : N ^ N I (3^ G G,)(V°"n G N) t{n) < 2^(i°sn) | 

Unless stated otherwise, in this paper, for each i e N, A represents any of the following 
classes of functions 

comp = { / : {0, 1}* — > {0, 1}* | / is computable } , 

Pi = { / : {0, 1}* — > {0, 1}* I / is computable in Gi time } , 
PiSpace = { / : {0, 1}* — > {0, 1}* | / is computable in Gi space } . 

For example, pospace is the set of all functions computable in linear space, p^ space, ab- 
breviated pspace, is the set of all functions computable in polynomial space, and Pj space 
is the set of all functions computable in space n^'"^"^ for some k E N. Given a class 
of functions A, let bound(A) C {t : N ^ N} denote the class of time or space bounds 
defining A: bound(comp) is the set of all computable functions t : N ^ N, and, for all 
i eN, bound(pi) = bound(pispace) = Gi. 

If D is a discrete domain, we say a function / : D — > M is /S.- computable if there is a 
function / : i:> x N ^ Q such that r)-f{x) \ < 2"'^ for all r e N and a; e il), and / e A 
(with r coded in unary and x and the output coded in binary). We say that / is exactly 
^-computable if f : D —>■ Q and / G A, and we say that / is dyadically ^-computable if 
/ : D ^ Q2 and / G A. 

2.4 Reductions 

Let Af be a Turing machine and S* G C. We say M computes S if, on input n G N, M 
outputs the string S \ n. We define an oracle Turing machine (OTM) to be a Turing 
machine M that can make constant-time queries to an oracle sequence, and we let OTM 
denote the set of all oracle Turing machines. For i? G C, we say M operates with oracle R 
if, whenever M makes a query to index n G N, the bit R[n] is returned. We write to 
denote the OTM M with oracle R. 
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Let S,R G C and M G OTM. We say S is Turing reducible to R via M , and we write 
S <T R via M, if computes sE In this case, define M{R) = S. We say S is Turing 
reducible to R, and we write S <t R, if there exists M e OTM such that 5* <t R via M. 
We say S is A-Turing reducible to R via M, and we write S <^ R via M, if computes 
S, and there is a function q e bound(A) such that, for all e N, M outputs S \ n using 
at most q{n) time or space (depending on the resource defining A). We say S is A-Turing 
reducible to R, and we write S <^ R, if there exists M G OTM such that 5* <^ R via M. 
We say S is Turing equivalent to R, and we write S =t R, if S <t R and R <t S, and we 
say S is A-Turing equivalent to R, and we write S =^ R, if S" <^ R and i? <t S. 

If A = comp, then a A-Turing reduction is nothing more than a truth-table reduction 
(see [48])|§ We write S <tt R to denote that S is truth-table reducible to R (i.e., that 
5* <^j?™P i?). If A = pjspace or pj, and we identify a sequence S E C with the language 
L C {0, 1}* for which S = xl, then a A-Turing reduction is an EjSPACE or Ej (see [31]) 
Turing reduction, respectively. Since this paper deals exclusively with sequences, we will 
use the convention of calling such a reduction a pjspace- or pj-Turing reduction, indicating 
that the polynomial bound is in terms of the length of the prefix of the characteristic 
sequence (the output) and not in terms of the length of the strings in the language (the 
input). 

Let S,P,ReC and Ms,Mp e OTM such that S <t P via Ms and P <t R via Mp. 
Define the composition of Ms with Mp, denoted Ms o Mp, to be the OTM that works 
as follows. On input n G N and with oracle i?, {Ms o Mp)^ simulates Mg to compute 
S \ n. Whenever a bit of P is queried by Ms, {Ms o Mp)^ simulates Mp for the minimum 
number of steps needed to compute that bit of P. 

Observation 2.3. <t is transitive: if S <t P via Ms and P <t R via Mp, then S <t R 
via Ms o Mp . 

3 Decompression and Dimension 
3.1 Decompression via Reductions 

Let S,R e C and M e OTM such that S <t R via M. Define #(S \ n, M^) to be the 
query usage of M^ on S \ n, the number of bits of R queried by M when computing the 
string S \ n. (If we instead define #(5 \ n, M^) to be the index of the rightmost bit of R 

"'^This differs from the more standard definition of a Turing reduction in that S \ n, instead of S[n], is 
computed on input n. The definitions are equivalent if at least polynomial time is allowed. 

truth-table reduction is typically defined to be a Turing reduction that halts on all oracles; it is easy 
to to see that this occurs if and only if the time (equivalently, space) used by the reduction on input n is 
bounded by a computable function of n. 
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queried by M when computing S \ n, all results of the present paper still hold.) Define 

PmW^R) = limmf , 



p'l^{S,R) = limsup' 



n 



Viewing R a.s a compressed version of S, pJ,j{S, R) and p^,f{S, R) are respectively the best- 
and worst-case compression ratios as M decompresses R into S. Note that < p^iS, R) < 
P'Ij{S,R) < oo. For 5* G C, the lower and upper Turing decompression ratios of S are 
respectively defined 

p-{S) = min {pIj{S,R) \ S <T R via. M} , 

MeOTM 

p^{S) = min {pIj{S,R) \ S <T Rvia M} . 

AfeOTM 

Note that < p~{S) < p^{S) < 1. As we will see, by Lemma [5.21 and Theorem 15.51 the 
two minima above exist. In fact, there is a single OTM M that achieves the minimum 
decompression ratio in each case. 

The lower and upper A-Turing decompression ratios of S are respectively defined 

p-^iS) = inf {p-^,iS,R)\ S<^ RviaM}, 
mgotm 

pI{S) = inf {pIj{S,R) \ S <^ Rvia M} . 

MeOTM 

Recall that a <I^™^-reduction is simply a truth-table reduction. Therefore, for all G C, 
the lower and upper truth-table decompression ratios of S are respectively defined 



PttiS) = Pcomp{S)= inf {pm{S, R) \ S <tt Rym M} , 

a/gotm 

pUS) = P+omp(^)= inf {ptAS,R)\ S<,,RviaM}. 

mgotm 

3.2 Effective Dimension 

See [33,34] for an introduction to the theory of effective dimension. 

Effective dimension was first defined in [33]. It is based on martingales, which are 
strategies for betting on bits of an infinite sequence. 

1. An s-gale is a function d : {0, 1}* — > [0, oo) such that, for all w G {0, 1}*, 

d{w) = 2-'[d{w0) + d{wl)]. 

2. A martingale is a 1-gale. 
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Intuitively, a martingale is a strategy for gambling in the following game. The gambler 

starts with some initial amount of capital (money) d(X), and it reads an infinite sequence 
S of bits. d{w) represents the capital the gambler has after reading the prefix w \Z S. 
Based on w, the gambler bets some fraction of its capital that the next bit will be and 
the remainder of its capital that the next bit will be 1. The capital bet on the bit that 
appears next is doubled, and the remaining capital is lost. The condition d{w) — '^("'°)+^("'^) 
ensures fairness: the martingale's expected capital after seeing the next bit, given that it 
has already seen the string w, is equal to its current capital. The fairness condition and 
an easy induction lead to the following observation. 

Observation 3.1. Let /c e N and let d : {0, 1}* [0, oo) be a martingale. Then 

J2 d{u) = 2^d{X). 
ue{o,i}'= 

An s-gale is a martingale in which the capital bet on the bit that occurred is multiplied 
by 2*, as opposed to simply 2, after each bit. The parameter s may be regarded as the 

unfairness of the betting environment] the lower the value of s, the faster money is taken 
away from the gambler. Let d : {0, 1}* [0, oo) be a martingale and let s G [0, oo). Define 
the s-gale induced by d, denoted d^^\ for all w E {0, 1}* by 

d^'\w)^2^'-^'>^''^d{w). 

If a gambler's martingale is given by d, then, for all s E [0, oo), its s-gale is d^^\ 

The following theorem, due to Lutz, establishes an upper bound on the number of 
strings on which an s-gale can perform well. 

Theorem 3.2 ( [33]). Let d be an s-gale. Then for all w E {0, 1}* , k eN, and a E R+, 

there are fewer than ^ strings u E {0, l}'^ for which 

max{2(^-^)H(i(^^^;)j > cid{w). 

CoroIIairy 3.3. Let d be a martingale. Then for all I E 'M., w E {0, 1}*, A; e N, and 
a E R"*"; there are fewer than ^ strings u E {0, l}'^ for which 

d{wu) > a2''-^d{w). 

Let S* G C, s G [0, oo), and let d : {0, 1}* [0, oo) be an s-gale. d succeeds on S, and 
we write S E S°°[d], if 

limsup(i(5' [ n) = oo. 

n—*oo 

d strongly succeeds on S, and we write S E S^j.[d], if 

liminf d{S f n) = oo. 

n— »oo 

The next lemma follows easily from the proof of the Exact Computation Lemma of [33]. 
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Lemma 3.4 ( [33]). If d is a A-computable s-gale and 2* is a dyadic rational, then there 
is a dyadically A-computable s-gale d such that S°°[c/] C S°°[(i] and S^j,[(i] C S^j.[(i]. 

Let d : {0,1}* [0, oo) be an s-gale. We say that d is constructive (a.k.a. lower 
semicomputahle, suhcomputahle) if there is a computable function d : {0, 1}* x N ^ Q such 
that, for all w G {0, 1}* and t G N, 

1. d{w, t) < d{w, t + 1) < d{w), and 



Let -R G C. We say that R is Martin-Lof random, and we write R G RAND, if there 
is no constructive martingale d such that R G S°°[(i]. This characterization of Martin- 
Lof randomness, due to Schnorr [44], is equivalent to Martin-Lof 's traditional definition 
(see [30,35]). 

If there is a martingale d that succeeds on a sequence 5 G C, then d makes arbitrarily 
high capital on 5*. Using a standard technique (cf. [37]), one may construct from d a 
martingale d' that strongly succeeds on S. This is done by maintaining a "side account" of 
capital that is not used to bet: i.e., the capital in that account is always allocated equally 
between and 1 when betting. Whenever d makes strictly more than $1, $1 is moved into 
the side account. Since d succeeds on S, it will eventually make more than $1 in the main 
account again, and so infinitely often, the side account will grow by $1, whence d' strongly 
succeeds on S. It is clear that if d is A-computable, then d' is also A-computable. 

Observation 3.5. Let S E C such that there is a A-computable martingale that succeeds 
on S . Then there is a A-computable martingale that strongly succeeds on S. 

This technique works for constructive dimension as well. Furthermore, Schnorr [44] 
showed that there is a universal constructive martingale that succeeds on every sequence 
not in RAND. This leads to the following observation. 

Observation 3.6. There is a constructive martingale d such that S^j,[d] = RAND'^. 

Let d : {0, 1}* x N ^ Q be the computable function testifying that d is constructive. 
The following theorem, due independently to Hitchcock and Fenner, states that d^*^ is 
"optimal" for the class of constructive t-gales whenever s > t. 

Theorem 3.7 ( [16,24]). Let s > t G [0, oo), and let d be a constructive t-gale. Then 



By Theorem 13.71 the following definition of constructive dimension is equivalent to the 
definitions given in [2,34]. Let 5" G C. The (constructive) dimension and the (constructive) 
strong dimension of S are respectively defined 



2. lim d{w, t) = d{w). 



t— >oo 



S°°[c/] C S°°[d(")] and ^Z[d] ^ 



dim(S') 
Dim(^) 



inf{s G [0,oo) I ^ G S°°[d(^)]}, 
inf{sG [0,oo) I 5GS^,[d(^)]}. 
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Intuitively, the constructive (strong) dimension of S is the most unfair betting environment 
s in which the optimal constructive gambler d (strongly) succeeds on S. 

Observation 3.8. If S e RAND, then dim(5) = Dim(5) = 1. 

The following theorem, due to Mayordomo [36], gives a useful characterization of the 
constructive dimension of a sequence in terms of Kolmogorov complexity, and it justifies 
the intuition that constructive dimension measures the density of algorithmic information 
in a sequence. 

Theorem 3.9 ( [36]). For all S eC, 

K{S \n) 



dim(iS') = liminf 



n 

and 



Dim(S') =limsup' ^ ^ 



n 

(s) 

Let G\ denote the set of all A-computable s-gales. For all S* G C, the A-dimension 
and the A-strong dimension of S are respectively defined 

dimA(^) = inf I s G [0, oo) (3d G G^'^) 5 G S°°[rf] } , 



and 



DimA(5') = inf I s G [0, oo) | (3d G G^^) S G SZ[d\ } 



We say S' G C is A-regular if dim a (5') = DimA(5'). 

The following alternate characterization of the space-bounded A-dimensions, resem- 
bling Theorem 13.91 is due to Hitchcock [22] . 

Theorem 3.10 ( [22]). LeH G N and A G {comp, p^space}. For all S 

dimA(5') = inf liminf ^ — ^- — ^, 

pebound(A) n^oo n 

and 

n- rc:^ • f r KS^(^ \ n) 
DimA^oj = mf hmsup . 

pebound(A) n-^oo n 

Let r,t G [0, oo) and S* G C. Note that if di and ^2 are martingales such that df^ 

(r) 

succeeds on S and ^2 strongly succeeds on S, then di and d2 can be combined into a 
single martingale d that simulates di and d2 in separate "accounts". Furthermore, if di 
and d2 are both A-computable, then d is also A-computable. This leads to the following 
observation. 

Observation 3.11. Let S E C, t > dimA(5'), and r > DimA(5'). Then there is a A- 
computable martingale d such that rf^*-* succeeds on S and rf'-'"-' strongly succeeds on S. 
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If is a A-computable s-gale, then d' : {0,1}* [0, oo) defined for all w G {0,1}* 
by d'{w) = d{w)/d{X) is also an s-gale. It is clear that d' is A-computable, and that d' 
(strongly) succeeds on the same sequences on which d (strongly) succeeds, whence the 
following holds. 

Observation 3.12. Let s > 0. For every A-computable s-gale d, there exists a A- 
computable s-gale d' such that d'{X) = 1, S°°[rf'] = S°°[rf], and S~[rf'] = S^Jrf]. 

The same technique cannot be used in general for a constructive s-gale d, since the 
initial capital d{X) may be uncomputable. However, by dividing each value d{w) by a 
rational approximation to d{X), we may assume that all constructive s-gales have initial 
capital arbitrarily close to 1. 

Observation 3.13. Let s > and e > 0. For every constructive s-gale d, there exists a 
constructive s-gale d' such that 1 < d\X) < 1 + e, S°°[rf'] = S°°[rf], and ^'^,[d'] = S^Jrf]. 

Though an analogue of Theorem 13.101 for pj-bounded Kolmogorov complexity and the 
Pj-dimensions is currently unknown, Lopez- Valdes and Mayordomo [51] have proved a 
characterization of polynomial-time dimension and strong dimension in terms of reversible 
polynomial-time compressors and decompressors. This characterization will be useful in 
constructing time-bounded dimension extractors. It is easily verified that their character- 
ization extends to the pj-dimensions for z > 1. 

Lopez- Valdes and Mayordomo characterize the pj-dimensions as follows. Let i G 
A Pi-compressor is a pair (C, D) of functions C : {0, 1}* — > {0, 1}* and D : {0, 1}* x N ^ 
{0, 1}*, each computable in pj time, such that, for all w G {0, 1}*, D{C{w), \w\) = w. A 
Pj-compressor does not start from scratch if, for all but finitely many G N and w G {0, 1}*, 

In particular, Lopez- Valdes and Mayordomo note that a pj-compressor (C, D) does not 
start from scratch if, for all w,u & {0,1}*, C{w) and C{wu) have a common prefix of 
length at least 

111 

\C{w)\- -^ + \og\u\. (3.1) 



Let VCi denote the set of all p^-compressors that do not start from scratch. 
Theorem 3.14 ( [51]). For all i e Z+ and S e C, 

n^oo 77, 



dimp^(5) =inf I {C,D)ePCi 
and 

Dimp,(^) =inf ( {C,D)eVCi 



y \C{S\n) 
hm sup ■ 



n 
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4 Dimension Extractors 



This section investigates various effective dimensions in wfiicfi effective reducibiiities may 
be used as extractors: constructive dimension, extracted witfi Turing reductions, com- 
putable dimension, extracted witfi trutfi-table reductions, pjSpace-dimension, extracted 
witli pjSpace-bounded Turing reductions, pj-dimension, extracted witli pj-bounded Tur- 
ing reductions, and finite-state dimension, extracted witli information lossless finite-state 
transducers. 

To understand the motivation for studying dimension extractors, it is helpful to un- 
derstand classical extractors. In general, an extractor is an algorithm used to transform a 
source of weak randomness into a source of stronger randomness. Extractors are motivated 
in part by the abundance of weak random sources in nature - for instance, electrical noise 
from Zener diodes - and the need for uniform (i.e., strong) random sources in probabilis- 
tic algorithms. Von Neumann's [53] coin flip technique is the simplest and most famous 
extractor: a biased coin may be used to simulate an unbiased coin by always flipping the 
coin twice, ignoring the combinations HH and TT, and interpreting HT to mean H and 
TH to mean T. 

More formally, in computational complexity theory, a extractor is a function, generally 
computable in polynomial time, taking as input a string drawn from a probability distri- 
bution X on {0, 1}" with min-entropy at least k, and a much smaller string of length d, 
called the seed, drawn from the uniform distribution on {0, 1}''. The extractor's output is 
"close" to uniformly distributed, but much longer than the seed. The min-entropy of X 
is defined min3,g{o,i}n logPrfx = it is the Shannon self-information [47] of the string 

with the highest probability in {0, 1}". If k is strictly between and n, X may be thought 
of as "partially random" ; n bits drawn from X have at least k bits of randomness. The 
goal of an extractor is to transform X into a distribution that is closer to "fully random" , 
i.e., to output m bits that have close to m bits of randomness. See [46] for a comprehensive 
survey of extractors in computational complexity. 

For algorithmic purposes, a deterministic infinite sequence that appears random to any 
algorithm often works just as well as a truly probabihstic source. The complexity class 
BPP, defined by Gill [19] to be those languages decidable by a randomized polynomial 
time algorithm with probability of correctness at least 2/3, is generally regarded as the set 
of decision problems feasibly decidable by a randomized algorithm. Bennett [4] (refining 
measure-theoretic arguments of [5] and [1]) has demonstrated that, given access to any 
oracle sequence that is algorithmically random in the sense of Martin-Lof, every language 
in BPP can be decided deterministically in polynomial time. Book, Lutz, and Wagner [8] 
have shown a wealth of similar characterizations of BPP and other randomized complexity 
classes in terms of oracle access to Martin-Lof random sequences. Lutz [32], using the 
techniques of resource-bounded measure [31], improved the result of Bennett by showing 
that all of BPP can be decided in polynomial time relative to any pspace-random oracle, 
which is a sequence that appears random to any polynomial-space-bounded algorithm. 

Recall that, given a resource bound A, A-dimension quantifies how close a sequence is 
to being A-random. Gu and Lutz [20] improved Lutz's above-mentioned pspace-random 
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oracle result by showing that all of BPP is polynomial time decidable relative to any 
oracle sequence with positive pspace-dimension (in fact, the oracle needs only have positive 
A3-dimension). Therefore, for certain applications, if a sequence has positive effective 
dimension, it contains sufficient algorithmic randomness to act as a substitute for a truly 
probabilistic source. This highlights the parallels between the theory of effective dimension 
and randomness extractors. A non-random sequence with positive dimension may be 
considered "weakly random": the first n bits of a sequence with Hausdorff and packing 
A-dimension equal to a contain about an bits of A-randomness. 

Reimann [41] and Terwijn asked the question, given any sequence S such that dim(S') > 
0, does oracle access to S allow us to compute a Martin-Lof random sequence? Miller and 
Nies [38] posed the related questions, does oracle access to S allow us to compute a sequence 
of constructive dimension 1, or arbitrarily close to 1, or strictly greater than dim(S')? View- 
ing constructive dimension as a quantification of the amount of randomness contained in a 
sequence, a computation increasing the constructive dimension of a sequence performs the 
same function as the extractors mentioned earlier: the computation transforms a partially 
random source into a more random source. Therefore Reimann, Terwijn, Miller and Nies 
are essentially asking whether Turing reductions can extract constructive dimension. 

4.1 Constructive Dimension Extractors 

We show that constructive dimension can be extracted in a weaker sense. Using Ryabko's 
result [42, 43] on optimal reversible compression of sequences, we demonstrate that, for 
every e > and every sequence S such that dim(5') > 0, there is a sequence P, Turing 
equivalent to S, such that Dim(P) > 1 — e. In fact, there is a single oracle Turing machine 
that accomplishes this extraction, taking a rational (3 > dim(5') as an input parameter 
used to control the size of e. Moreover, the extractor uses close to an optimal number of 
bits of the input sequence to compute the output sequence, in the sense that for infinitely 
many n, about n bits are required from S to compute dim(5') ■ n bits of P. 
The next theorem is due to Ryabko [42,43]. 

Theorem 4.1 ( [42,43]). There exist OTMs Mg and M^, with Me taking a single input 
/3 G Q, with the property that, for every S E C and every rational [3 > dim(5'), there exists 
P eC such that P <t S via M^{p), S <t P via Md, and Pm^{S, P) < (3. 

The next theorem states that any sequence in which almost every prefix has Kolmogorov 
rate bounded away from zero can be used to compute a sequence with infinitely many 
prefixes of nearly maximal Kolmogorov rate. Furthermore, this can be done with a single 
OTM taking a rational upper bound on the constructive dimension of the input sequence. 

Theorem 4.2. There exists an OTM Me, with Me taking P E Q as input, such that, for 
all S E C such that dim(S') > 0, and all e > such that < /? — dim(S') < e ■ dim(S')/3, 
there exists P G C such that P <x S via Me{f3), S <t P, and Dim(P) > 1 — e. 

Laurent Bienvenu [6] has shown that the bound of /3 — dim(S') < e ■ dim(S')/3 required 
in the hypothesis of Theorem 14.21 can be improved to (3 — dim(S') < e ■ dim(S'). 
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The statement of Theorem 14.21 is comphcated by the rational input j3 required to make 
the OTM Me uniform over all sequences. The following corollary states simply that Turing 
reductions can extract constructive strong dimension from positive constructive dimension. 

Corollary 4.3. For each e > and each S* G C such that dim(5') > 0, there exists P G C 
such that P =T "S* and Dim(P) > 1 — e. 

Proof of Theorem \4.2\ Let S, f3, and e be as in the statement of the theorem, and define 
6 = /3- dim(^) > 0. Let M^, G OTM be as in Theorem SH so that Mf computes 
P, M[ computes S, and, letting Up = #(5* \ n, M[) for all i G N, since {S, P) < 



lim inf — < /3 



K{S \ m) ^ Up 



^ {3°°n G N) liminf ^ — ' — > ^-6 by Theorem ED 

m^oo m n 

^ (3°°n G N) K(5 t n) > rip - 5n. (4.1) 
Since 5<e- dim(5)/3 < e ■ dim(5)/2, 

(V°°n G N) K{S \ n) > -^5n. (4.2) 

Ryabko's construction of Md is such that entire prefixes of the oracle sequence are queried 
at once: whenever the bit at index i G N is queried, all bits j < i are also queried. 
Thus, a program M simulating with the first rip bits of P can calculate S \ n, and 
M can be encoded in |enc(P \ np)\ + 0(1) bits. Thus there is a constant c such that 
K{S \ n) < Up + 2 log Tip + c, which together with (14.21) implies that 

(V°°nGN) (5n< I (np + 2 log rip + c). (4.3) 

Combining flO) and fl43|) . 

{3^n eN)K{S \n)>np-^{pn + 2 log Up + c). (4.4) 

If the OTMs Me and M^ are defined as in Theorem 14.11 then Me(/9) and M^ testify that 
P =T S. It remains to show that Dim(P) > 1 — e. Suppose for the sake of contradiction 
that Dim(P) < 1 — e. Then it would be the case that 

(V°°m G N) K(P \m) <m-em. (4.5) 

Since dim(S') > 0, 5* is uncomputable, and therefore rip grows unboundedly with n. A 
program that produces P \ rip can be used in conjunction with M^ to produce S \ n. 
Therefore, for a suitable constant c' ~ \Md\, 

(V°^n G N) K(^ \ n) < K(P \ Up) + c 

< Up — erip + d 

< np-^ {pn + 2 log np + c). (4.6) 
But (HH) contradicts Hence, Dim(P) > 1 - e. □ 
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Unfortunately, technique of the preceding proof does not show that Turing reductions 
are able to increase constructive dimension in addition to constructive strong dimension 
(i.e., that almost every prefix of the output sequence has high Kolmogorov complexity). 
Nies and Reimann [39] have shown that constructive dimension cannot be extracted with 
weak truth-table reductions (a Turing reduction in which the query usage on input n is 
bounded by a computable function of n): for every rational a G [0, 1], there is a sequence 
5* such that dim(S') = a, and every sequence P that is weak truth-table reducible to S 
satisfies dim(P) < a. Since the Turing reduction in our proof is also a weak truth-table 
reduction, it cannot be the case that dim(P) > dim(5'). 

Bienvenu, Doty, and Stephan [7] have shown that the stronger results of the next sub- 
section hold for constructive dimension as well, using weak truth-table reductions, thereby 
improving the present paper's Corollary 14.31 to be as strong as Theorem 14.81 (and in fact 
stronger, as a uniform extractor is exhibited similar to that in Theorem 14. 21) . However, it is 
also shown in [7] that there is no uniform Turing reduction is capable of always increasing 
constructive dimension by a fixed amount: for every a, (3 with < a < (3 < 1, and every 
Turing reduction M, there is a sequence S such that dim(5') > a and, if computes the 
sequence R, then dim(i?) < jS. This implies that the reduction techniques of [7] and of the 
present paper, which use no property of the input sequence other than a simple bound on 
its dimension, cannot be used to prove the existence of constructive dimension extractors. 

Buhrman, Fortnow, Newman, and Vereshchagin [10] and Fortnow, Hitchcock, Pavan, 
Vinodchandran, and Wang [17] have demonstrated related constructions for extracting 
Kolmogorov complexity from finite strings. Buhrman, Fortnow, Newman, and Vereshcha- 
gin show that there is an efficient algorithm, taking as input any non-random string, that 
outputs a small list of strings of the same length as the input string, where at least one 
output string is guaranteed to have higher Kolmogorov complexity than the input. Note 
that given a finite string x and the value K(x), an algorithmically random string containing 
exactly the amount of algorithmic information in x may be extracted from x: namely, a 
shortest program for x. The value K(a;) - requiring at most log bits to represent - may 
be considered "advice" bits that help the algorithm extract randomness from x. Fortnow, 
Hitchcock, Pavan, Vinodchandran, and Wang improve upon this observation by showing 
that there is an efficient algorithm such that, for any a, (3 such that < a < /3 < 1, if 
the input string x has Kolmogorov complexity at least a\x\, then, given a constant (with 
respect to a and /?) number of advice bits, the output string y (with \y\ = Q{\x\)) will 
have Kolmogorov complexity at least f3\y\. This is shown to hold for space-bounded Kol- 
mogorov complexity as well. The advice bits are necessary; Vereshchagin and Vyugin [52] 
have shown that no uniform algorithm is capable of extracting Kolmogorov complexity 
from finite strings. 

4.2 Other Effective Dimension Extractors 

As noted, Ryabko's result leads easily to an extractor for constructive dimension. In con- 
trast, our main extractor result. Theorem 14. 8 1 relies on new techniques to exhibit extractors 
for resource-bounded A-dimension [33], where A represents the class comp of computable 
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functions or, for each i E N, any of the classes p^space of pjSpace-computable functions or 
the classes p^ of pj-time computable functions. We show that for every e > and every 
sequence S such that dimA(>S') > 0, there is a sequence P, A-Turing equivalent to S, such 
that DimA(P) > 1 — e. We show a similar result for finite-state dimension, with the ex- 
tractor implemented by an information lossless finite-state transducer [26,47]. In contrast 
to the constructive dimension extractor, a different reduction machine is required for each 
sequence 5*. 

In addition to the near-optimal extraction of strong dimension, the extractors for com- 
putable, pispace. Pi, and finite-state dimension are shown to partially extract dimension 
as well. More precisely, if the input sequence has dimension d and strong dimension D, 
then the sequence output by the extractor has dimension at least d/D — e, where e is 
an arbitrarily small positive constant. Therefore, for any regular sequence - a sequence 
whose dimension and strong dimension agree - dimension is nearly optimally extracted, in 
addition to strong dimension. 

4.2.1 A-Dimension Extractors 

This subsection examines dimension extractors for computable dimension, space-bounded 
dimension, and time-bounded dimension. 

An OTM that computes a sequence S, together with a finite prefix of the oracle that it 
queries, is a program to produce a prefix of S. Thus, the query usage of a space-bounded 
OTM on that prefix of S cannot be far below the space-bounded Kolmogorov complexity 
of the prefix of S. This is formalized in the following lemma, which bounds the optimal 
compression ratio below by dimension. 

Lemma 4.4. Let i e N and A e {comp, pjspace}. For all S e C, 

PAi^) > dimA(-S), and pXiS) > DimA(-S). 

Proof. Let 5, P e C, and let M e OTM such that S <^ P via M. For P = S, S <^ P 
via the trivial "bit-copier" OTM that always queries exactly n bits of P to compute n bits 
of S, so we may assume that for all n e N, ^{S \ n, M^) < n. Thus, since M has available 
at least a linear amount of space, we may assume that each bit of P is queried at most 
once and cached, and that subsequent queries are retrieved from the cache. 

Let ttm be a self-delimiting program for M. Let Pn E {0, ) be the oracle bits 
of P queried by M on input n, in the order in which they are queried. Recall the self- 
delimiting encoding function enc. For each n e N, let tt^ = 7rM'7rMenc(n)enc(pn), where 
77 M' is a self-delimiting program that simulates M, encoded by ttm, on input n, encoded 
by enc(n), with oracle P, encoded by enc(p„). When M makes its i^^ query to a bit of P, 
the bit Pnii] is returned. Since M queries each bit of P at most once, the bit from p„ will 
be correct, no matter what index was queried by M, since the bits of Pn are arranged in 
the order in which M makes its queries. 

Then C/(7r„) — S f n, so if there exists s G bound(A) such that M uses at most s{n) 
space on input n, there exists q e bound(A) such that, for all n e N, KS^(5' \ n) < |7rn|. 
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By Theorem 13.101 



dimA(5') 



i}ebound{A) n^oo n 




< 



< 




vtm'VTmI +logn + 21oglogri + #(5 f n, M^) + 21og#(5 \n,M'" 



) + 6 



PMiS,P), 



whence dim(S') < p^{S). Similarly, Dim(S') < 



plis). 



□ 



The following theorem is used to construct A-dimension extractors, and to give new 
characterizations of these dimensions for A = comp or p^space. The theorem also holds for 
Pi dimension, but the second application, the characterization of dimension in terms of op- 
timal sequence decompression, seems to require a pj-time-bounded Kolmogorov complexity 
characterization of p, dimension, which at the present time is unknown. Furthermore, the 
theorem only holds for polynomial time and above, and does not include linear time. 

Theorem 4.5. Let i eN and A G {comp, pjspace, pj+i}. For all S E C and 6 > 0, there 
is a sequence P G C and an OTM M such that 



Proof idea: If the A-dimension of 5* is small, then a A-computable martingale d performs 
well on S. Thus, if we have already computed a prefix 5* f of S*, then on average, d 
increases its capital more on the next k bits of S than it would on other k-hit strings 
that could extend S \ n. This places the next k bits of 5* in a small (on average) subset 
of {0, l}'^, namely, those strings on which d increases its capital above a certain rational 
threshold c, which is chosen to be slightly smaller than d{S \ (n+k)), the amount of capital 
made after the next k bits of S. Since d is A-computable, it is possible to enumerate strings 
from this small set by evaluating d in parallel on all possible length-/c extensions of S \ n, 
and outputting a string u G {0, 1}^ whenever d{{S \ n)u) is greater than c. We will encode 
the next k bits of S as an index into this set, where the index will represent the order in 
which this parallel evaluation enumerates the string we want - the next k bits of S. This 
technique is similar to that used by Merkle and Mihailovic [37] to prove Theorem 15. 4[ 

We require two lemmas to prove Theorem 14.51 Lemma 14.71 shows that the average 
number of bits needed to encode the index of a length-/c extension of S" f n is close to 



1 



S =^ P, with S<^ P via M. 



2. 



Pl,{S,P)<diu,^{S) + 5. 



3. 



pl,{S,P)<DimA{S) + 6. 
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the dimension of S times k. We will also need to encode the threshold c into the oracle 
sequence. Lemma 14.61 shows that we can find a rational threshold c that requires so few 
bits to represent that it will not affect the compression ratio when added to the oracle 
sequence, yet which is still a close enough approximation to d{S \ {n + k)) to keep the 
index length of Lemma 14.71 small. 

In the following lemma, intuitively, the values fcj, n^, rj, and Cj respectively represent 
the number of bits in the i^^ block of the sequence S, the number of bits in the first i blocks, 
the value of a martingale after reading the first bits, and a rational approximation to 
that value from below. 

Lemma 4.6. There exists a constant C such that the following holds. Let N G Z+. Let 

ko = nQ = 2. For all i G Z'^ , let ki = \N log Ui^i] and Ui = ni^i + ki. Lei G [1, 2"*] fl Q2. 

Then for all hut finitely many z G N, there exists Ci G Q2 such that ri {l — j < Ci < Vi 

and K'^"'(q) < ki ■ 2/N. Furthermore, such a program can be computed from ki and in 
at most Cui steps. 

Proof. We prove the cases kf < < 2"''"'"^ and 1 < < kf separately. 

Suppose that kf < < 2"'"'"^. In this case we will choose Cj to be an integer. Set 
m G Z+ such that 2"-^ < kf < 2"^. Since > kf > 2"*"^ [logri] > m - 1. 

If Tj is an integer ending in at least [logr^] — m bits, let Cj = rj — 1. Otherwise, let 
Cj G Z+ be the integer whose binary representation is xO^^"^^^^~"^, where x G {0, 1}™ is the 
first m bits of [r^J . Since Cj shares its first m bits with or is equal to — 1, 



so 1^1 — jjj < d < Ti. Ci can be fully described by the first m bits of rj, along with the 

binary representation of the number [log r^] — m of O's that follow. Thus, describing Cj 
requires no more than 



bits, for all but finitely many i G N. 

Now suppose that 1 < Vi < kf. We let q approximate by the binary integer [rj , 
plus a finite prefix of the bits to the right of r^'s decimal point in binary form. If x.S is 
the binary representation of r,, where x G {0, 1}* and 5 G C, let Cj G be represented 
by x.y, where y ^ S. Then r j — Cj < 2~I^L 

Since < kf, \x\ < logkf = 21ogfci. We need — ci < Vi/kf. Since — Ci < 
2^l'^l, it suffices to choose y ^ S such that 2~l^l < Vi/kf, or \y\ > \og{kf/ri). Let \y\ = 
\\og{kf /vi)] < 2 log ki, since r, > 1. Thus + \y\ < 41ogfcj, so describing Cj requires at 
most 41ogA;j bits. 



o [log ri] -»7i _ 



1 < 





m + log([logrj] 



ITT') < log k^ + log rii 

< logk^ + \og2''^+'/^ 

< 2ki/N 
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If this results in Ci = Vi, rather than Ci < Vi, then is already a dyadic rational requiring 
no more than \y\ bits to the right of the decimal point. In this case, let Cj = rj — 2^1^' > 

^1 — p-j instead. 

Let 7r(cj) be one of the two programs just described for computing Cj from r^. It is clear 
that vr(cj) runs in 0{ni) time. We now demonstrate that 7r(cj) itself can be computed from 
i and in 0{ni) time. 7r(Q) is simply constructed from initial bits of in either case. It 
follows that in the first case, 7r(cj) can be created in 0{ni) time, since logr^ < n^, and the 
first case uses only the integral part of r^. In the second case, we have already shown that 
the integral part x of Cj and the fractional part y of q each consist of O(logfcj) bits of rj. 
Therefore, in the second case, vr(cj) can be created in 0(nj) time. □ 

For t G M, c e Q, s G {0, 1}*, A; G N, and : {0, 1}* [0, oo) a t-gale, define 

41 = { « e {0, 1}^ I d{su) > c } (4.7) 

to be the set of all length-/c extensions of s on which d makes at least c capital. The 
following lemma shows that l^^^csl small on average if d makes a lot of capital on a 
sequence beginning with prefix s, if c is close to the capital that d has after reading k bits 
beyond s. 

Lemma 4.7. Let S G C, r > t > 0, and let d be a martingale such that d^^^ succeeds on 
S and d^^'^ strongly succeeds on S. Write S = SoSiS2 . . where, for all i E N, ki = |sj|, 
and rii = |so • • • s^l- Let q G M satisfy d{S \ ni)g{i) < Ci < d{S \ rii), where g{i) G (0, 1) 
satisfies — ^*=2 ^oS5'(i) — oijii) as i oo. Then 

E]=o log 



lim sup 

i—>oo 

and, if ki = oijii) as i oo, then 



d,Cj ,S t"j — 1 



< r, {U 



E^=o log 



■'j=0 

lim inf 



A 



d,Cj ,S \nj — i 



< t, (4.9) 



Proof We first show that (l4l9!) holds. Let t' > t, and, for all i G N, let At = A]^^^^^g^^^_^. 

It suffices to show that, for infinitely many i G N, X]j=olog|^il — '^'^j- Since d^^^ succeeds 
on S, for infinitely many G N, 

d{S \ n) >2^'-'^''d{\). (4.10) 

A martingale can at most double its capital after every bit, and each index n with rij < 
n < Ui+i is at most ki bits beyond rij. It follows that for infinitely many i G N, 

d(5 f rii) > 2(^-*)">-'=>rf(A). (4.11) 
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For all i G N, set G M such that ^(5" \ ni) = 2''^ ''^d{S \ By induction on 



d{S \ rii) = d{X)Y[2''^-^^ . (4.12) 

j=0 

Then, by equations (I4.1ip and (I4.12p . and the fact that Yl]=o^i — infinitely many 

i e N, 

i i 

-Q 2kj-h > 2(i-*)">-'^« =^ ^{kj - h) > (1 - - 

j=0 j=0 

i 

=^ Ij < trii + ki. 

j=0 

Recall that q > d{S \ ni)g{i) = g{i)2^^'^^d{S \ nj_i). By Corollary [3l3] (take k = ki,l = 
li,a = 1 — = S \ nj_i) and the definition of k, it follows that \Ai\ < 2'-^/g{i), and so 
\og\Ai\ <li- log g{i). Let Co,i = logl^ol +log|Ai| - /o - ^1- Then 



log \Aj\ < ^Ij-Yl log ^(^) + co,i 



j=0 



< tUi + ki-^ log 5'(0 + Co,: 



i=2 

i 

= t'ni + (t-t>i + /ci-^log^(i) + co,i. (4.13) 

i=2 

t < t', ki = o{ni), and J2]=2 log5'(0 = o{ni), so for infinitely many i, ^*=olog l^jl — '^'^i- 
The proof of (14. 8 p is similar, replacing "for infinitely many i" conditions with "for all 
but finitely many i." The only difference is that fl4.1Up holds for all but finitely many n, 
and so there is no need to derive (14. lip . Consequently, the term ki does not appear on 
the right-hand side of (I4.13p . and so the condition ki = o{ni) is not necessary to show that 
fOjl holds. □ 

Let t G M, c e Q, s G {0, 1}*, k E N, and let d : {0, 1}* [0, oo) be a t-gale. If d is 
exactly A-computable and u G A^^l^, then the following procedure computes the index of 
u in a lexicographical ordering of ^4^''^^^. 

indg,. («e{0,ir) 

1 i' ^0 

2 for each m' G {0, l}'^ in lexicographical order 

3 do if d{su') > c 

4 then it u' = u 

5 then output i' and exit 

6 else i' -I— i' + 1 
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If M ^ Af^l^^, ind^'^^ ,,(«) is undefined. For all u G {0, 1}'', ind^^] , 



{u)< 
(fc) 



A 



(k) 

d,c,s 



when it is 



defined. The computation of str^j.^ : N ^ {0, 1} , the inverse of ind)^^^, is similar: 



str 



(k) 



1 ^ 

2 for each u' G {0, 1}^ in lexicographical order 

3 do if d{su') > c 

4 then if i' = i 

5 then output u' and exit 
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else 



i' + l 



Both ind^'^^g and str^^^^^ are uniformly ^-computable for all d, c, s, and A;, in the sense that 
each may be implemented by a single A-bounded Turing machine taking d, c, s, and k as 
auxiliary input, provided d is exactly A-computable. 



Proof of Theorem \4.5[ If dimA(5') = 1, then the trivial "bit-copier" OTM M suffices to 
compute P = S, where p]^{S,P) = pj^{S,P) = dimA(5') = DimA(5') = 1, so assume that 
dimA(5') < 1. 

Let t,r E Q2 such that dimA(5') < t < dimA(5') + S and DimA(5') < r < DimA(5') + S. 
Then by Observation 13 . 1 ll there is a A-computable martingale d such that d^^^ succeeds on 
S and d^^'^ strongly succeeds on 5*. By Observation 13.51 dimA(S') < 1 implies that we may 
assume that d strongly succeeds on 5*. By Lemma [331 we may assume that d is dyadically 
A-computable. By Observation 13. 121 we may assume that d{X) = 1. 

2 2 

max 



Let N = 
i G Z+, let kj 



dimA(5') + 6 ■ 
[A^logni_i] and 



r' DimA{S) + 5-t 
= + ki. Write 5* 



. Let ko = uq = 2. For all 
. . ., where, for all i G N, 



ki = and nj = |s'o. . By Observation 13.11 d{S \ rii) < 2^' for all i G N. Since d 
strongly succeeds on S, for all but finitely many i, d{S \ rii) > 1. Let C G N be as in 
Lemma [4.61 For all such i G N, taking = d{S \ rii), choose Cj G for i, a, and as 
in Lemma [4. 6[ and let 7r(ci) represent a program testifying that K'-^"*(ci) < ki ■ 2/N, which 
can be computed from i, d{X), and d{S \ rii) in at most Crii steps. 
Let P = P0P1P2 • • • , where, for all i E N, 



Pi = enc (indj;^ 5^„^_^(si 



71 (d). 



Because str^*^'.'' a. is an inverse of ind^,'^''' qt„ , we can write each Sj as 



Since ind^*^^^^ ^5|.„._^, str)^^^^^^^^^._^ and d are all A-computable, P =^ S. Note in particular 
that, since ki = O(lognj), the search over all strings of length ki done in the computation of 
ind^'^*^^ and str^''^^^^ requires searching only a polynomial (in Ui) number of strings. 
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Let M be the OTM such that computes S. It suffices to show that pJ.[{S,P) < 
dimA(5') + 6 and p\.^{S,P) < DimA(5') + 5. Recall that ki = rii — nj_i, and ki grows 
unboundedly with i, so i = o(nj) as i —>■ oo. Then 

-^logfl-l) = ^[21og/.,-log(A;, + l)-log(A;,-l)] 

i 

< 2 5^[logfc,-log(fc,-l)] 

i=2 

i 

< 2 J2 [log(iVlog(n,_i) + 1) - log(iVlog(n,_i) - 1)] 

i=2 

= 0{i) = o{ni), 

since linij^oo [log(A^log(nj„i) + 1) — log(A^ log(r;,j_i) — 1)] = 0. Thus g{j) = 1 — -^ satisfies 
the requirement — Yl]=2^'^S9{j) = o{ni) in the hypothesis of Lemma [4.71 Then 

+ rQP^ r #(^rn,M^) \n.,Mn ■ v. ( ^ 

pJf[o, F) = limsup = limsup since ki = o[ni) 

72— >oo ^ i — >oo 

Ei=o enc (mdj;!]_5^„._^(sj)) 71(9) 
= lim sup 



lim sup 



< limsup 




Ej=o logins; 



d,Cj,S\nj — i V'^J 



rii 



E}=o logind^, 




< 2/N + r < dimA(S') + 6 by Lemma 1131 

Similarly, Pm{S, P) < DimA(5') + 6. □ 

We now arrive at the main result of this section. It is a A-bounded extension and 
improvement of Corollary 14. 3[ It states that A-strong dimension may be extracted, using 
A-Turing reductions, from any sequence of positive A-dimension, for A = comp, p^space 
(for i > 0), or Pi (for i > 1). Furthermore, Theorem 14.81 achieves a partial extraction of 
dimension in addition to strong dimension. The reduction is not uniform: the OTMs used 
depend on the sequence from which dimension is being extracted. The case of Theorem 
l4.8l for A = comp was independently discovered by Laurent Bienvenu [6]. Theorem 14.81 has 
been extended to constructive dimension by Bienvenu, Doty, and Stephan [7]. 



Theorem 4.8. Let i eN and A G {comp, p^space, pj+i}. For each e > and each S E C 



such that dimA(5') > 0, there exists P E C such that P =^ S, dimA(-P) > ~ ^' 



and DimA(-P) > 1 — e. 
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Proof. Let i G N and A G {comp, pjspace}. We first prove this case, and at the end of the 
proof we handle the case A = pj for i G Z"*". 

Let 5* and e be as in the statement of the theorem. Let < 5 < e ■ dimA(>S')/2. Choose 
P e C, Md e OTM, and Me G OTM for S and 6 as in Theorem KB such that P S 
and, letting Up = \ n, ) for all n G N, 

liminf — < dimA(5') + 5 

KS'^iS \ m) n 
=^ inf lim inf > lim mi — — 5 by Theorem 13.101 

<jebound(A) m^oo m n^oo n 

KS'^(S' \ m) n 
=^ (Vg G bound (A)) liminf > liminf — — 5 

=^ (Vg G bound(A))(3~n G N) liminf ^ ' ^ > ^ - 5 

m-^oo m n 

=^ (Vg G bound(A))(3~n G N) KS«(5 \n)>np- 6n. (4.14) 
Since 5 < e ■ dimA(5')/2, by Theorem 13.101 

(Vg G bound(A))(V°^r2 G N) KS'?(5 \ n) > -^5n. (4.15) 

Note that the construction of in the proof of Theorem 14.51 is such that entire prefixes 
of the oracle sequence are queried at once: whenever the bit at index i G N is queried, 
all bits j < i are also queried. Thus, a program M' simulating Md with the first np bits 
of P can calculate S \ n, and M' can be encoded in |enc(P \ np)\ + 0(1) bits. Thus 
there is a constant c and, since Md uses A space, there exists t G bound(A) such that 
KS*(S' \ n) < Up + 2 log np + c, which together with 04.151) implies that 

(V°°n EN) 5n<^{np + 2 log Up + c). (4. 16) 

Combining fHlij) and fHl6D . 

(Vg G bound(A))(3°°n G M) KS''(^ \ n) > Hp - ^ (p„ + 21ognp + c) . (4.17) 

Suppose for the sake of contradiction that DimA(-P) < 1 — e. Then by Theorem I3.10[ it 
would be the case that 



(3s G bound(A))(V°°m G N) KS'(P \ m) < m - em. (4.18) 

Since dimA(5') > 0, S* is not A-computable (see [33], Lemma 4.13), and since S is A- 
computable by Md with access to P, Up must grow unboundedly with n. A program that 
produces P \ npin s{np) space can be used in conjunction with Md (which uses at most t{n) 
space) to produce S f n in at most t{n) + s{np) space. For the case A = comp, t{n) + s{np) 
is bounded by a computable function of n. By part[3]of Theorem l4.5l Up = 0{n), so, for the 
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case A = p^space, the space bound n i— » t(n) + s{np) is contained in Gi = bound (p^space). 
Then for a suitable constant c' ~ \Md\, 

(3gGbound(A))(V°°neN) KS^(5 fn) < KS'{P\np) + c' 

< Up — eUp + d 

< np-^{pn + 2\ognp + c). (4.19) 

But (Km contradicts fHTTD . Hence, DimA(P) > 1 - e. 

To see that dimA(P) > ^^ifs] - e, let < < dimA(^), D' > D > DimA(5'), and 
< 6 < min{dimA(5') — d,D ~ DimA(>S')}. (Note that our previous requirement that 
< 6 < e ■ dimA(5')/2 does not contradict this.) 

Let q G bound(A). It suffices to prove that, for all but finitely many n, KS^^'^{P \ n) > 
-^n, where r G bound(A) is given below. 

By Theorem SSI P =^ S and pjj^{S, P) < D. Recall that for each n G N, = #(5 \ 
n, M[). Then there exists r G bound(A) such that, for all but finitely many n, 

dn < KS%S \ n) < KS"+^(P \np)+2 logn. (4.20) 

The first inequality holds because d < dimA(5'), and the second because M^^'' together 
with a binary encoding of n (requiring less than 2 log n bits) is a program computable in 
r{n) + q{n) space to output S \ n. 

By the fact that p^j^{S,P) < D, the assumption that D' > D, and (I4.20p . for all but 
finitely many n, 

D' 

Dn > Up =^ Dn — 2—\ogn>np 

d 

=^ dn — 2 log n > ^'^p 

=^ KS^+^(P\np)>^np. (4.21) 

Recall from the proof of Theorem 14.51 that the length of the i^^ block in S, is o(i), 
and that \pi\, the length of the i^^ block of P, is at most |sj| + o(|sj|). This implies that 
\pi\ = oijip) if Pi occurs on prefix n^. Thus, for all n, \np — (n + l)p| = oirip). Therefore, 
f l4.2ip implies that, for all but finitely many n G N, KS'''*'^(P \ n) > -^n. This proves the 
theorem for A G {comp, p^space}. 

We next show that the theorem holds for A = pj, where i G Z^. Note that the OTMs 
used in the proof of Theorem 14.51 compute P from 5* (respectively, 5* from P) using S \ n 
to compute P \ Up (respectively, P \ Up to compute S* f (n — 1)), where, for all G N, 

(n + l)p-np = 0(logn), (4.22) 

because the blocks in which S and P are processed have length logarithmic in the length 
of the prefixes they extend. This implies that there exist pj-computable functions C : 
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{0, 1}* ^ {0, 1}* and D : {0, 1}* x N ^ {0, 1}* such that, for all neN, D{C{S \ n),n) = 
S \ n; C simply encodes however many whole blocks of S fit into S* f n as the corresponding 
blocks from P, and the last partial block of S may simply be copied bit-for-bit. Equation 
fl4.22p tells us that the last block's length is negligible, which implies that (C, D) achieves 
a compression ratio asymptotically as good as that of Theorem 14.51 Therefore 

lim inf ^^^^ ^""^^ < dimp, (S) + 6 (4.23) 

n—*oo Tl 

and 

lim sup ^""^^ < Dimp, {S) + 6. (4.24) 

Equation (14.221) also implies that (C, D) satisfies (13.11) and therefore, (C, D) is a p,- 
compressor that does not start from scratch. Although Lemma 14.41 is not known to 
hold for Pi-dimension, Theorem 13.141 and inequalities (14.231) and (14.241) tell us that (C, D) 
achieves a compression ratio within 5 of the optimal compression ratio achievable by pj- 
compressors that do not start from scratch. Hence, by an incompressibility argument 
similar to that given in the first part of the proof (and similar to the proof of Theorem 
dimp,(P) > - e, and Dimp,(P) > 1 - e. □ 

Corollary 4.9. Let i G N and A G {comp, p^space, pj+i}. For each e > and each 
A-regular S* G C such that dimA(5') > 0, there exists P G C such that P =^ S and 
dimA(P) > 1 - e. 

Fortnow, Hitchcock, Pavan, Vinodchandran, and Wang [17] have shown a similar ex- 
traction result for constructive, computable, and space-bounded strong dimension, which 
starts from the weaker hypothesis that only the strong dimension of the input sequence is 
positive. Furthermore, the extractor runs in polynomial time. Lemma 4.2 of [17] requires 
advice and is stated only for pspace dimension, but can be improved to eliminate the ad- 
vice [25] . The technique also works for any dimension having a space-bounded Kolmogorov 
complexity characterization, leading to the following theorem. 

Theorem 4.10 ( [17]). Let i eN and A G {comp, pj space}. For each e > and S E C, 

1. //Dim(S') > 0, then there exists P G C such that P =^ S and Dim(P) > 1 — e. 

2. //DimA(5') > 0, then there exists P G C such that P =E^ S and DimA(P) > 1 — e. 



4.2.2 Finite-State Dimension Extractors 

We next show that information lossless finite-state transducers can extract finite-state 
dimension [12]. Finite-state dimension has multiple equivalent definitions [9,12,14,23]. 
For this paper, we exclusively use the characterization given in [12] in terms of information 
lossless finite-state transducers. 

A finite-state transducer (FST) is a 4-tuple T = (Q, 5, q^), where 
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• Q is a nonempty, finite set of states, 



• 5 : Q X {0,1} ^ Q is the transition function, 

• u : Q X {0,1} {0, 1}* is tlie output function. 



• 50 e Q is the initial state. 

For all X e {0, 1}* and b e {0, 1}, define the extended transition function 5 : {0,1}* ^ Q 
by the recursion 

^(A) = go, 

S{xb) = S(S{x),b). 

For X G {0,1}*, we define the output of T on a; to be the string T{x) defined by the 
recursion 

T(A) = A, 

T{xb) = T{x)v(5{x),b) 

for all X E {0, 1}* and b G {0, 1}. 

A FST can trivially act as an "optimal compressor" by outputting A on every transition 
arrow, but this is a useless compressor, because the input cannot be recovered. A FST 
T — [Q, 5, u, qq) is information lossless (IL) if the function x ^ {T{x), S{x)) is one-to-one; 
i.e., if the output and final state of T on input x uniquely identify x. An information 
lossless finite-state transducer (ILFST) is a FST that is IL. We write ILFST to denote the 
set of all information lossless finite-state transducers. 

Let 5* G C. The finite- state dimension [12] and the finite- state strong dimension [2] of 
S are respectively defined 

dimFs(-S)= inf hminfi^^^^^-^, 

^ ' CelLFST n-^oo n 

and 

DimFs(5)= inf limsup '^^'^ ^ ""^l 



CglLFST n 

Intuitively, the finite-state dimension (resp. strong dimension) of a sequence represents 
the optimal best-case (resp. worst-case) compression ratio achievable on the sequence with 
any ILFST. We say 5* G C is FS-regular if dinipsl'S') = DimFs(>S'). 

Given a sequence P G C and a FST T, define T(P) to be the output of T on P, the 
shortest element G C U {0, 1}* such that, for all n E N, T{P \ n) ^ S. Let S,P E C 
and T G ILFST. We say S is IL- finite- state reducible to P via T, and we write S <ilfs P 
via T, if T{P) = S. We say S is IL- finite- state reducible to P, and we write S <ilfs P, if 
there exists T G ILFST such that S <ilfs P via T. We say S is IL-finite-state equivalent 
to P, and we write S =ilfs P: if -S" <ilfs P and P <ilfs -S". 

The following well-known theorem [26,27] shows that each ILFST T computes a function 
T : C ^ C whose inverse is computable by another ILFST. 
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Theorem 4.11 ( [26,27]). For all T e ILFST, there exists T'^ E ILFST such that, for 
alls eC, T-^{T{S)) = S. 

Corollary 4.12. For all S,P eC, S <ilfs P if and only if S =ilfs P- 

The following lemma, which is similar to Observation 13.111 follows easily from Lemma 
5.6 of [9]. It shows that a single ILFST can always be found whose compression ratio 
simultaneously approximates both of the optimal compression ratios that dimps and Dimps 
represent. 

Lemma 4.13 ( [9]). For all S E C and all 6 > 0, there exists an ILFST C such that 

liniinfH^^M<dimFs(5) + 5, 

and 

limsup ^""^^ < DimFs(5) + 5, 

The following theorem shows that ILFST's can extract finite-state dimension from 
sequences in a similar manner to Theorem 14. 8[ 

Theorem 4.14. For each e > and each S E C such that dimFs(5') > 0, there exists 
PeC such that P =iLFs S, dimFs(P) > - e, and DimFs(P) > 1 - e. 

Proof. Let S and e be as in the statement of the theorem. Let 6 = dimFs(S') • e/3. Choose 
C G ILFST for S and 6 as in Lemma 14.131 so that liminf < dimFs(5') + 6 and 

limsup < DimFs(5) + 6. Let P = C{S), and let Up = \C{S \ n)\ for all neN. Let 

a = dimFs(5') ■ 2/3. Since dimFs(5') > 0, for all but finitely many n E N, Up > an. Then 

lim inf — < dimFs(5') + S 

n— >oo Tl 

=^ inf lim inf - — ^ — ^- — ^ > lim inf — — 5 

C'elLFST m^oo m n-^oo n 

=^ (VC G ILFST) liminf "^^'^^ ^""^^ > liminf ^ - 5 

=^ (VC G ILFST) (3~n G N) liminf ^^'^^ ^ "^^^ >^-6 

m^oo rri n 

=^ (VC G ILFST) (3~n G N) \C'{S \n)\>np-6n 

=^ {yC G ILFST) (3~n G N) \C'{S \n)\>np- aen. (4.25) 

Let C2 G ILFST. Let C3 = C2 o C denote the composition of C2 with C, the ILFST 
such that, for every x G {0, 1}*, Csix) = C2(C(x)). It is well-known [26,27] that ILFST's 
can be composed in this manner. By (I4.25p . for infinitely many n G N, 

Up — aen < \C3{S \ n)\ 

= \C2iC{S\n))\ 



\C2{P \n: 



V) 
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Thus, dividing both sides by Up, since Up > an for all n, 

\C2{P\n)\ \C2{P\np) 
iim sup > lim sup 

n— >oo IT' n— >oo ^p 

n 

> 1 — limsupae — 

n—*oo TT'p 

> 1 -e. 

Since C2 was arbitrary, this establishes that DimFs(P) > 1 — e. 

dimFs(S) 

DimFslS) 



We now show that dimFs(-P) > — e. In a similar derivation to that giving 



lim sup — < DimFs(5') + 6 



n— >oo 



n 



, 1- np DimFs(5) 

=^ limsup — < dimFs(o)-r: 7777 + d 

n^oo n dimFs(o) 

=^ inf lim inf — ^ — — ^— > — — \ ] (lim sup — — 6 

C'elLFST m^oo m DimFs(o) \ n^oo n 

=^ (VC G ILFST)(V°°r2 G N) \C'{S \ n)\ > ^™Fs(>gh _ ^^ ^e) 

DimFs(^) 

By (14.261) . for all but finitely many n G N, 

^^K-aen) < \C,iS I n)\ 

= \C2{C{S\n))\ 
= \C2{P\np)\. 

Since C is an ILFST, there is a constant c > such that, for all n, ((n + l)p — rip) < c. 
Since C2 is an ILFST, the same reasoning implies that there is a constant c' G N such that, 
for all n G N, \C2iP \ (n + l)p)\ - \C2iP \ np)\ < c'. Therefore 

n^oo n n^oo Up 



DimFs(S') \ ri^oo Up 

^ dimFsl'S') _ 
~ DimFsl^) 

Since C2 was arbitrary, this establishes that dimFs(-P) > DimFs(g) ~ ^' 

Clearly, P <ilfs S via C. By Corollary |4l2l P =ilfs □ 

Corollary 4.15. For each e > and each FS-regular 5 G C snc/i i/iai dimFs(>S') > 0, there 
exists P G C such that P =ilfs S and dimFs(-P) > 1 — e- 
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5 Every Sequence is Optimally Decompressible from 
a Random One 

Kucera [28,29] and Gacs [18] independently obtained the surprising result that every se- 
quence is Turing reducible to a Martin-Lof random sequence. In the words of Gacs, "it 
permits us to view even very pathological sequences as the result of the combination of 
two relatively well-understood processes: the completely chaotic outcome of coin-tossing, 
and a transducer algorithm." Merkle and Mihailovic [37] have provided a simpler proof 
of this result using martingales, which are strategies for gambling on successive bits of a 
sequence. 

Bennett [4] claims that "This is the infinite analog of the far more obvious fact that 
every finite string is computable from an algorithmically random string (e.g., its minimal 
program)." However, the analogy is incomplete. Not only is every string s computable 
from a random string r, but r is an optimally compact representation of s. Gacs showed 
that his reduction achieves a decompression ratio of 1: for any n, n + o{n) bits of R are 
required to compute n bits of S. But as in the case of strings, sequences that are sparse 
in information content should in principle be derivable from a more compact description. 
Consider the following example. It is well known that K, the characteristic sequence of the 
halting language, has constructive dimension and constructive strong dimension [3]. The 
binary representation of Chaitin's halting probability = '^^j ^aits 2~'*^' (where M ranges 
over all halting programs and \M\ is M's binary description length) is an algorithmically 
random sequence [11]. It is known that K <t ^ (see [30]). Furthermore, only the first n 
bits of fl are required to compute the first 2^ bits of K, so the asymptotic decompression 
ratio of this reduction is 0. Q can be considered an optimally compressed representation 
of K, and it is no coincidence that the decompression ratio of achieved by the reduction 
is precisely the constructive dimension of K. 

We generalize this phenomenon to arbitrary sequences, extending the result of Kucera 
and Gacs by pushing the decompression ratio of the reduction down to its optimal lower 
bound. Thus, we complete Bennett's above-mentioned analogy between reductions to ran- 
dom sequences and reductions to random strings. We show that, for every sequence S, 
there is a sequence R such that S <t R, where the best-case decompression ratio of the 
reduction is the constructive dimension of S, and the worst-case decompression ratio is 
the constructive strong dimension of S. Furthermore, we show that the sequence R can 
be chosen to be Martin-Lof random, although the randomness of R is easily obtained by 
invoking the construction of Gacs in a black-box fashion. Finally, a single machine works 
in all cases; in analogy to the universal Turing machine used to define Kolmogorov com- 
plexity, a single Turing reduction reproduces each sequence S from its shortest description. 
This result gives a way to associate with each sequence S another sequence R that is an 
optimally compressed representation of S, decompressible by a single, universal reduction 
machine. As in the case of Kolmogorov complexity, the compression direction is in gen- 
eral uncomputable; it is not always the case that R <t S. This result also extends a 
compression result of Ryabko [42,43], discussed in section |2l although it is not a strict im- 
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provement, since Ryabko considered two-way reductions (Turing equivalence) rather than 
one-way reductions. 

The following lemma shows two senses in which the composition of two oracle Turing 
machines in a transitive Turing reduction bounds the compression ratio of the transitive 
reduction below the product of the compression ratios of the two original reductions. 

Lemma 5.1. Let S,P,ReC and Ms, Mp G OTM such that S <t P via Mg and P <t R 
via Mp, and let M = Ms o Mp, so that S <t R via M. Then 

pt,iS,R)<ptj^iS,P)pt,^iP,R), 

and 

Pl,iS,R)<p,,^iS,P)pt,^iP,R). 

Proof. Let rg~^ > p|j^(S', P), r^^ > pJj^{S,P), and rf+ > pjj^{P,R). It suffices to show 
that p+ (5, R) < r^+r^+ and R) < r^-r^+. 

For infinitely many n, #(5* \ n,Mg) < r^~n. For all but finitely many n, #(S' \ 
n, Mg) < rs~^n, and if{P \ n, Mp) < Tp^n. Then, for all but finitely many n, to compute 
S \ n, M^ requires 

#(S r n, M^) = # (P r # r n, Ml') , M|) < r«+# [S \ n, M^) < r^+r^+n 
queries to R. Since this holds for all but finitely many n, 

PIj[S,R) = limsup < r^^Vp. 

For infinitely many n, to compute S \ n, M requires 

#(5 r n, M^) = # (P r # (5 r n, M^) , M|) < r^+# {S \ n, M^) < r^T^+n 
queries to R. Since this holds for infinitely many n, 

Pa AS, R) = limmf < Vp. 

n^oo fl 

□ 

The following may be proven using the same technique as the proof of Lemma 14.41 

Lemma 5.2. Let S,ReC and M E OTM such that S <t R via M. Then 

p^.j{S,R) > dim(S'), and p]^j{S,R) > Dim(S'). 

The next lemma is similar to Lemma 14. 6[ but it achieves a more compact encoding of 
the rational number at the expense of a worse approximation to the real it is representing. 
This extra compactness is required in order to make the length of the program for the 
rational number asymptotically smaller than the length of the block in which it resides, so 
that its length can be entirely discounted in the limit (not just made an arbitrarily small 
percentage of the block length, as in Lemma [4.61) . 
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Lemma 5.3. There exists a constant C G N such that the following holds. For all i G N, 
let Tj G [1,2* ]. Then for all but finitely many i G N, there is a rational number Ci G 
such that Tj (l — ^) < Ci < and K(cj) < Clogi. 

Proof. We prove the cases > and 1 < rj < separately. Suppose Vi > i"^. In this 
case we will choose q to be an integer. Set m G Z"*" such that 2"*"^ < < 2"^. Since 
ri>i^> 2''"~\ Rogrj] > m — 1. 

Let Q G Z+ be the integer whose binary representation is xO^'"^'''^"'", where x G {0, 1}™" 
is the first m bits of [rj . Since q shares its first m bits with rj, 

r,-c,< 2ri°g'-'l— - 1 < !l±^ - 1< 5, 

so rj (l — < Q < rj. Cj can be fully described by the first m bits of r^, along with the 
binary representation of the number [logr^] — m of O's that follow. Thus, describing q 
requires no more than m + log( [log rj] —m) < logi^ + 1 + loglogc + logi^ = O(logi) bits. 

This will not work if G and rj's least significant [logrj] — m bits are 0, which 
would result in c, = r^, rather than Cj < r^. In this case, let 



1 = numa (rep2(num2(x) - l)iri°g'-«l-™) 



where num2(x) is the integer whose binary representation is x, and rep2(7^) is the binary 
representation (with possible leading zeroes) of n G N. This likewise requires O(logz) bits 
to describe. Since > i^, Q = — 1 > {l — 

Now suppose that 1 < < i"^. We approximate by the binary integer [r^J, plus a 
finite prefix of the bits to the right of r^'s decimal point in binary form. If x.S is the binary 
representation of r^, where x G {0, 1}* and S G C, let q G Z"*" be represented by x.y, where 
ynS. 

Since < i^, |x| < logi^ = 0(log2). We need — Cj < p- for q to approximate 
closely. Since — q < 2~l^l, it suffices to choose y \Z S such that 2~l^l < ^, or |?/| > log ^. 

Let |?/| = log^ = 0(log2), since > 1. Thus \x\ + \y\ = O(logi), so describing q 
requires 0(\ogi) bits. 

This will not work if is a dyadic rational x.z, where x, 2; G {0,1}* and \z\ < \y\, 
which would result in Cj = r,, rather than Cj < r^. In this case, let r'- G [r^ (l — ) , ^i) be 
irrational. Choose q for r'- by the method just described, such that r'- > Ci > r[ {l — 2^), 
and Cj requires 0(log(z\/2)) = 0(log2) bits. Then Q > (l — r^) by the triangle inequality, 
and Ci <r[ < r^. □ 

The next theorem says that every sequence is Turing reducible to a random sequence. 
Part [1] is due independently to Kucera and Gacs, and part [2] is due to Gacs. 
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Theorem 5.4 ( [18,28,29]). There is an OTM Mg such that, for all S E C, there is a 
sequence R G RAND such that 

1. S <T R via Mg. 

2. pI,^{S,R) = 1. 

The next theorem is the main result of this section. It shows that the compression 
lower bounds of Lemma [5.21 are achievable, and that a single OTM M suffices to carry out 
the reduction, no matter which sequence S is being computed. Furthermore, the oracle 
sequence R to which S reduces can be made Martin-Lof random. The randomness of R 
is easily accomplished by invoking the construction of Gacs in a black-box fashion; the 
majority of the work in the proof is establishing the bound on the compression. Therefore, 
the next theorem improves the query usage bound of Theorem 15.41 but it does not provide 
a new proof of Theorem 15. 4[ The proof is similar to the proof of Theorem 14.51 However, 
due to the existence of the optimal constructive martingale d and the fact that we require 
only the decompression direction to be computable, we need not approximate the optimal 
decompression ratio from above; we can actually achieve it using d. 

Theorem 5.5. There is an OTM M such that, for all S G C, there is a sequence R e 
RAND such that 

1. S <T R via M. 

2. Pm{S,R) =dim(^). 

3. pti{S,R) = Dim(5). 

Proof. Parts [1] and [3] are shown. The proof for part [2] is similar to part [3l 

If S* G RAND, then S <t S via the trivial "bit copier" machine M', with lower and 
upper compression ratio dim(S') = Dim(S') = 1, so assume that S ^ RAND. 

A single OTM M" suffices to carry out the reduction described below, no matter what 
sequence S ^ RAND is being computed. If 5 G RAND, then M' is used. These two 
separate reductions are easily combined into one by reducing each sequence 5 to a random 
sequence bR via M G OTM, where b e {0,1}, R = S if S e RAND, and R is given by the 
construction below if S* ^ RAND. The bit b indicates to M whether to use M' or M" for 
the reduction. Hence a single OTM M implements the "optimal decompression" . 

For all i G N, define ki = i + 1, and define uq = 1 and rij = nj_i + fcj = for 
i > 0. Note that rii < for all sufficiently large i. Write 5* = Sq^i • • such that, for all 
i G N, = ki] ki represents the length of the i^^ block into which we subdivide S. Ui is 
then |so • • • the total length of the first i + 1 blocks. For all i G N, define q G Q+ to be 
a rational number satisfying 

1. d{S \ Hi) (l — r^) < Ci < d{S \ Ui); i.e., q is a rational number approximating 
d{S \ Ui) from below. 
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2. K(q) = o{ki) as 2 — s> oo; i.e. q can be computed from a program asymptotically 
smaller than the length of the i^^ block. 

By Observation [3131 1 < d(A) < 2. By Observation [SH d{S \ n,) < 2"»d(A) < for 
sufficiently large i. By Observation 13.61 S ^ RAND implies that for all but finitely many 
i, d{S \ rii) > 1. Thus, by Lemma [573] (take rj = d{S \ rii)), there is a q G Q"*" satisfying 
the above two conditions. 

For all i G N, define the set Ai = Al^^l^g^^^_^ C {0, l}'^' as in Lemma [121 by 

Ai = {ue{0, 1}'^ I d{{S \ ni_^)u) > a} , 

the set of all length-Zcj extensions of S \ rti^i that add more capital to the optimal con- 
structive martingale d than SjZ — 1] does, using Cj as an approximation to d(5' \ rii). Since 
d(S' \ Hi) > Ci, it follows that Sj G Aj. 

For all i E N, let G N be the output of the following partial computable procedure, 
when given as input the string Sj — 1] G {0, 1}^*: 

1 Ai^ 

2 for t = 0,1,2,... 

3 do for each u G {0, 1}''' — Ai 

4 do if d((5' \ ni_i)u,t) > a 

5 then add u to Ai 

6 if M = Sj 

7 then output \Ai\ and halt 

In other words, Pi is the order in which d{S \ Ui) is shown to exceed Cj (i.e., to belong to 
Ai) by a parallel evaluation of d{{S \ ni^i)u, t) on all extensions u G {0, 1}''' of S \ rii-i, for 
t = 0, 1, 2, . . .. Since q < d{S \ rii), there exists some t G N such that d{S \ rii, t) > Ci, and 
so Pi is well-defined. The computation of str^^'^|.„._^, the inverse of ind[.^'_^|.^._^, resembles 
that of indj]^„^_^: 

1 Ai^ 

2 for t = 0,1, 2,... 

3 do for each u G {0, 1}'''- - Ai 

4 do it d{{S \ ni_i)u,t) > Ci 

5 then add u to Ai 

6 if \Ai\ = Pi 

7 then output u and halt 

Note that str^^'^|.^ will not halt if given as input an integer greater than \Ai\, and 

(k) 

ind\ will not halt if given a string that is not an element of Ai. 
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ind^^'].|.„._^ and str|.*^'^|.^._^ are similar to ind^^^ ,, and str^*^] ,,, respectively, but are de- 
signed explicitly to work with the lower-semicomputable martingale d. Instead of enu- 
merating strings in lexicographic order, they use the fact that the lower graph of d is 
computably enumerable via d to enumerate strings in the set Ai. 

For all i E N, let 7r(ci) denote a self-delimiting, shortest program for computing q. 
Define the sequence P G C by 

P = enc(po)7i"(co)enc(pi)7r(ci)enc(p2)vr(c2) .... 

Define the oracle Turing machine Ms that produces n bits of S, with oracle P, as follows. 
Let i{n) denote the block in which n resides - the unique z G N such that rii < n < rij+i. 
First, Mg reads the first i{n) + 1 blocks of P: 

enc(po)7r(co) . . . enc(pj(„))7r(Q(„)). 

Mg then calculates the first i{n) + 1 blocks of S iteratively. On block i, Mg first computes 
Pi from enc(pi) and q from 7r(cj). Then, Mg evaluates str^*^'_5|.^._^(pj) to obtain % and 
outputs it as the i^^ block of S. 

Since Sj G A, it follows that Pi < \Ai\, and so |enc(pi)| < log|ylj| + 21oglog|Aj| + 3. 
Note that 



j=2 ^ •> / j=2 ■> 



= -Y. + 1) + - 1) - 2 log j) 

= - log 1 + log2 + log? - log(z + 1), 

which converges to 1 as z — > oo, so (^(j) = 1 — satisfies — ^*=2 logfi'(j) = whence 
the conditions of Lemma 14.71 are satisfied. Therefore, by Lemma 14. 7i 

hmsup < hmsup — < Dim(i>). l^.l) 

By our choice of q, |vr(cj)| = o{ki), so Ej=o = as z ^ oo, giving 

Ej=olenc(pj)7r(9)| E}=o |enc(pj)| 
hmsup — = hmsup — . (5.2) 



Since Hi = M^±l)^ = o{ni), 



so 



E}=olenc(Pj)vr(cj)| E}=o |enc(Pj)vr(cj)| 
hmsup — < hmsup — . (5.-3) 
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In other words, because the block size grows slower than the prefix length, the lim sup over 
all blocks is at least the lim sup over all bits (and they are in fact equal by the definition 
of lim sup). 

For all n G N, Mg requires X]j=o bits of P in order to compute n bits of 

5", and hence, by fl5.ip - fl5.3l) . 

+ (Q V E;=o|encfe)7r(c,)| 
pJ.j^{b,F) = hmsup — < Uim(D). 

ra— »oo IT' 



Choose R G RAND for P as in the construction of Gacs in his proof of Theorem 15.41 
satisfying P <t R via Mg and plj^iP, R) = 1. Let M" = Ms o Mg. Then S <t R via M" 
and, by Lemma [5. 11 

p+„(5,i?) < pt.iS,P)pUP,R) < Dim(^). 



By Lemma [521 PM"iS,R) > Dim(5'). Similarly, pj^,p,{S,R) = dim(S'). □ 

Note that the block lengths used in the proof grow with the square root of the prefix 
length; if we write S = 'so'si ... as in the proof of Theorem 15.51 then for all i G N, 

= 0(a/|so • • • ^il)' similarly for the blocks of R. Bienvenu [6] has shown that this 
can be improved to = 0(log |so . . . 

It is instructive to compare Theorem 15.51 with Ryabko's Theorem 14. 1[ While Ryabko's 
theorem represents 5* with a more compact sequence R, it is not optimally compact, as 
a different decoding machine is required to get the compression ratio closer and closer 
to the optimal ratio of dim(S'). However, the major difference between the theorems is 
that Ryabko's construction does not achieve the bound between p"*" and Dim. Intuitively, 
Ryabko's theorem states that S may be compressed to a sequence R, where infinitely often 
(but not almost everywhere), approximately the first K{S \ n) bits of R suffice to produce 
S \ n. However, Ryabko's construction requires that the block lengths grow exponentially: 
for all i G N, |so . . . > 2*|so • • - Sjl. Therefore, while the lower compression ratio p~ is 
close to optimal in Ryabko's theorem, the upper compression ratio p"*" is infinite. 



6 Dimension Characterizations 

We use the decompression results of previous sections to characterize constructive dimen- 
sion as the optimal decompression ratio achievable on a sequence with Turing reductions, 
to characterize computable dimension as the optimal decompression ratio achievable on 
the sequence with truth-table reductions, and to characterize pjspace-dimension as the 
optimal decompression ratio achievable on the sequence with pj space-computable Turing 
reductions. See [14] for a collection of similar compression and decompression characteri- 
zations of finite-state dimension. 
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Theorem 6.1. For alii and S e C, 



dim(S') 
Dim(S) 




dimcomp(5') 
Dimcomp(5') 
dimp^spaco(5') 
Dinip^spacclS) 



Ppi space 



iS), 
{S). 



Proof. This follows immediately from Theorems 14.51 and 15.51 and Lemmas 14.41 and 15. 2[ □ 

Note that p^-dimension is absent from Theorem 16. 11 because Lemma 1131 is not known to 
hold for pj-dimension without a Kolmogorov complexity characterization of pj-dimension. 
Theorem I3.14[ however, characterizes pj-dimension in terms of optimal pj-computable de- 
compression if pj-computable compression (i.e., reversibility) is also required. 

These decompression characterizations differ from Mayordomo's and Hitchcock's Kol- 
mogorov complexity characterizations of these dimensions in that the compressed version 
of a prefix of S does not change drastically from one prefix to the next, as it would in the 
case of Kolmogorov complexity. While the theory of Kolmogorov complexity assigns to 
each finite string an optimally compact representation of that string - its shortest program 
- this does not easily allow us to compactly represent an infinite sequence with another 
infinite sequence. This contrasts, for example, the notions of finite-state compression [26] 
or Lempel-Ziv compression [54], which are monotonic: for all strings x and y, x implies 
that C{x) C C{y), where C{x) is the compressed version of x. Monotonicity enables these 
compression algorithms to encode and decode an infinite sequence - or in a more applied 
setting, a data stream of unknown length - online, without needing to reach the end of 
the data before starting. However, if we let ir^ and tTj, respectively be shortest programs 
for X and y, then x ^ y does not imply that vTa- □ vr^jfl In fact, it may be the case that 
TTx is longer than iiy, or that tt^ and TCy do not even share any prefixes in common. In the 
self- delimiting formulation of Kolmogorov complexity, tt^ cannot be a prefix of iTy. 

The Turing reductions of Theorems 15.41 14. H and 15.51 satisfy the stronger properties of 
the weak truth-table reduction (see [48]), which is a Turing reduction in which the query 
usage of the OTM on input n is bounded by a computable function of n. Thus, constructive 
dimension and strong dimension could also be defined in terms of decompression via weak 
truth-table reductions (i.e., dim(S') = p^tti^) and Dim(S') = Pwtt('S'))- 
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•^Monotone complexity [45,55] does not resolve this, since the universal monotone machine guarantees 
only the converse implication: C vTy x Q y. 
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